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1 . INTRODUCTION 


Let  X^,  X^, . . . be  a sequence  of  independent,  identically  distributed 
random  variables  taking  on  values  in  some  measurable  space  X . Let  Pj^,...,P 


K 


be  a finite  collection  of  probability  measures  on  X , and  let  H >k  = 1,...,K 

K 


denote  the  liypothesis  that  the  common  distribution  of  the  X 's  is  P,  . 

n k 


We  wish  to  associate  with  the  observed  sequence  X^,  a sequence  of 

decisions  d ,d  , , d 6 {H  , ,H,,}  about  the  true  hypothesis  H.  However, 

i z n i K 

the  decision  d^  at  time  n is  allowed  to  depend  on  X^,...,X^  only  through 
a finite-valued  statistic  T^€  {l,...,m}  , which  represents  the  current  state 
of  the  memory.  This  statistic  is  updated  after  each  observation,  i.e.. 


Vi  ■ • "--2 


where  f ; {l,...,m}  x X ->■  {l,...,m}  is  a time-invariant  updating  rule.  The 


decision  is  then  given  by 


d^  = d(T^)  , n=l,2 


where  d : (l,...,m}  ->■  } is  a time-invariant  decision  rule. 

1 K. 


Let  for  a given  f and  d 


Ff>(f,d)  . li.i  ! 


(1.1) 


N-HJo  n=l 


be  the  asymptotic  expected  frequency  of  incorrect  decisions  if  the  true 
hypothesis  is  . Our  goal  is  to  find  (f,d)  which  minimizes 


P (f,d)  = max  P^'^^(f,d) 
^ k=l,...,K  ^ 


(1.2) 


1 


I 


I 


I 


1 


(Thus  we  have  chosen  the  raj.nimax  criterion.  An  alternate  approach  would  be 


to  minimize 


P^(jr;(f,d))  = I TTj^P^’^^f.d) 


where  ti  = (Ti^,...,7r  ) is  a prior  distribution  on  {H  , ...,H  } . In  this 
IK.  IK 

report  we  consider  the  former.) 

The  pair  (f,d)  together  with  the  domains  and  ranges  of  the  two  functions 
is  formally  equivalent  to  the  definition  of  a finite  automaton  (see  e.g.  [1]). 
The  automaton  has  S = {l,...,m}  as  its  state  space,  X as  its  input  space, 


{H, } as  its  output  space,  and  f and  d as  its  state-transition 
i K 

function  and  state-output  function  respectively.  If  the  sequence  Xj^,X2».. 
of  i.i.d.  random  variables  is  applied  to  the  input  of  such  ^n  automaton  the 
resulting  sequence  of  states  T^,T2,...  is  then  a time-homogeneous  Markov 
chain  with  transition  probabilities 


?ij  = Pj^({x€  X : f(i,x)  = j})  , i,j£  S . 


(1.3) 


Hence  the  limit  in  (1.1)  always  exists.  If  the  state-transition  function  f 
is  such  that  the  resulting  chain  is  regular  then  in  fact 


P^‘"^f,d)  = M^(d  \Hj^))  , 


where  p is  the  stationary  distribution  on  S . Throughout  this  paper  we 
K 

assume  that  this  is  the  case,  i.e.,  we  consider  only  transition  functions  which 
yield  regular  Markov  chains  under  each  hypothesis. 

Following  Heilman  and  Cover  [3]  we  would  like  to  include  the  possibility 
that  the  transition  function  f can  be  randomized.  One  way  of  defining  such 
a randomization  would  be  to  Introduce  another  input  sequence  Yj^,Y2»...  of 
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i.i.d.  random  variables,  Independent  of  the  sequence  and  uniformly 

distributed  on  the  Interval  [0,1],  The  transition  probabilities  (1.3)  would 
then  be 

= E^ip^j(X)}  , where  (l.A) 

p^j(x)  = X({y€  [0,1]  : f(x,y,l)  = j})  , (1.5) 

A being  Lebesgue  measure  on  [0,1]. 

However,  we  find  It  more  convenient  to  express  the  randomized  state  transition 
function  f as  a pair  (A,  A)  as  follows: 

A “ { j • 1“1 ,...,ro  ^ J~l,...,m  ^ l^j}  , 

where  A, . are  measurable  subsets  of  X . 
ij 


^ = { : 1=1 , . . . ,m  ; 


where  6,,  2 0 and  6 < 1 for  all  l,j  . 

J " 

The  transition  probabilities  (1.5)  If  X = x 


j l,...,m  J j } , 


Is  observed  are  now  defined 


by 


Plj  (x)  “ whenever  9 x for  l^^j  , 

p (x)  = 1 - 5]  P^.(x)  , 

li  j/1 


and  (1.3)  becomes 


’ij  ^k^^lj^'^lj 


11 


= 1-  I 


'Ij 


We  will  refer  to  the  triplet  (A, A,  d) 
and  to  the  set  A as  randomization. 


(1.6) 


as  randomized  finite  automaton  (RFA) 


3 


Notice  the  class  of  all  randomization  is  closed  with  respect  to  multi- 


plication of  corresponding  elements,  that  is  if  A=  {6..}  and  A'=  {6'  } 

ij  xj 

are  randomization  then  AA'  = {6  6'  } is  again  a randomization.  Notice 

ij  ij 

also  that  the  sets  A.,  need  not  be  disiolnt. 

ij 

We  now  present  a simple  lemma  to  be  used  in  the  reniaining  sections. 
Lemma  1 : Let  (A,A,d)  be  a RFA,  let  , k = 1 , . . , ,K  , be  stationary 
distributions  of  the  resulting  Markov  chain  of  states.  Let  R = [r  ] be  a 

K 

K X K matrix  with  positive  entries 


let  o(A,A,d)  be  the  maximal  eigenvalue  of  R . 


P (A, A ,d)  > 1 - 


p(A,a  , d)  ' 


and  tliere  exists  a randomization  A'  such  that 


P^(A,AA  d)  = 1 - -7^~rc  . 

e p(A,A  ,d) 


Proof : 


(1-P  ) ^ = (1-max  P 

G G 

k 

max  (1-P  ^ = max  (b,  (d  ^(H  )))  ^ 

k ® k 


= ^ - P > 

k £=1 

since  by  Perron-Frobenius  theorem  the  maximal  eigenvalue  of  a positive  matrix 
can  never  exceed  the  largest  of  the  row-sums. 


Wv 

fe-  ■ 


To  prove  the  second  statement  let  v = (v^,...,v  ) be  an  eigenvector 

X K 

corresponding  to  p(A,A  ,d)  and  normalized  such  that 


\ > 0 , k = 1 , . . . ,K  , v^  + v^  + • • • + Vj^  = a , 

where  0 < a < 1 is  an  arbitrary  constant.  (This  is  always  possible  since  the 
matrix  R is  positive.)  Define  the  randomization  a'  = 

<5..  = — » , where  u = v whenever  itd  ^(H,  ) . 

^ £ IK  k 

(k)  (k) 

Let  p..  and  p’  be  transition  probabilities  and  p and  p'  the  sta- 
ij  ij  k 

tionary  distribution  of  (A,A,d)  and  (A,  A A ' ,d)  respectively.  Then  by 
(1.6)  for  any  k and  i?^j 

n-(k)  _ A. 

ij  ’ 

and  hence  for  any  partition  of  the  state  space  S into  two  subsets  and 


we  must  have 


I I p'^.u)  = I I 

q i/cq  ^<cq  ^cq 


1€S^ 


i€S^  j€S^ 


n j A r> 


i€S^  i€S^  “i 


i€S^  j€S^  “i 


Thus  ^ ^ with  > 0 independent  of  i so  that 


for  all  k,f,  = 1,...,K  . But  then  for  all  k 


p(A,A  ,d) 


since  V 


K 

I 

i-1 

Is  an  eigenvector  of 


Kit  \ 

p(A,A,d)  . 


Q.E.D. 


2.  UPPER  BOUND  ON  THE  ERROR  PROBABILITY  FOR  THE  3-HYPOTHESES  3-STATE  PROBLEM 
Let  K = 3 , m = 3 , and  let 


P*  = Inf  P^(A,A  ,d)  , 
e e 

the  Inflmum  being  taken  over  the  class  of  all  3-state  RFA.  Let  for  l,j  •=  1,2,3 


sup  P (A)/P  (A) 
ACX  ^ J 

^Ij  * Inf  P,(A)/P.(A) 
A'lX  ^ J 


(2.1) 


let 

g = (1/3)(Yj^2  ''^23  ^13^  ’ 

5 = (l/2)max{Y^2  ’'^23  ’ ^12  ^13  ’ ^23  ^13^  • 

In  this  section  we  show  that 

P*  < 1 - (1  + 2g  cosh(l/3  argcosh  G g . (2.2) 

We  also  establish  a simpler  but  looser  bound,  namely 

P*  < 1 - (1  + 2G^^^  )~^  . (2.3) 

e 

Notice  that  (2J)  Implies  that  If  Yj^2  * ^23  * ^13  " then  P*  = 0 , a result 

obtained  by  Sagalowlcz  In  [4]  and  extended  later  by  Yakowltz  In  [5]. 
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i 


m 


Proof  of  (2.2)  and  (2.3): 

Let  the  letters  i,j,k  stand  for  a permutation  of  1,2,3,  let  e > 0 


Let  A^^(e)  be  measurable  subsets  of  X such  that 


P.(A  (e))  P (A) 

i-ij < in£  + E , 


Pj(Ajj(c)) 


A 


let 


* . * 

Y.  . (e)  = ^ 


P,(A,,(e))  P.(A.^(c)) 


ij 


Pl(Ai.(£))  P.(A.^(e)) 


so  that  Y^j  £ ->■  0 . Consider  a 3-state  RFA  (A, A ,d)  , where 


A - {A..(e)}  , ^ = {6  } such  that  6 =6  = 0 , otherwise  arbitrary,  and 

iJ  iJ  ik  ki 


d(i)  = , d(j)  = H.  , d(k)  - 


P(A. .)6, . 
ij  ij 


C!©. 


P(A.  ^6.. 
Ji  Ji 


The  stationary  distribution  of  the  resulting  Markov  chain  of  states  are 
(see  Appendix  A) 


W(l)  - cP(A.^)8.jP(A^j)5^j 
P(j)  - cP(Ay)5^.P(A^.)6^, 


p(k)  = cP(Aj,)8j,P(A^)6,.  , 


where  c is  a normalizing  constant.  (The  epsilon  has  been  dropped  temporarily 
to  ease  the  notation.)  The  matrix  R of  Lemma  1 is  given  by 
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I 


■'4 

/■'’S' 


P.(A.  J«S. 


P.(A, .)6. . 
J ij  ij 


^<V^| 


\Ak>«3k 


and  its  characteristic  equation  has  the  form 


where 


(]-A)  - (l-X)C  + D •■=  0 , 

e e 


C . y-1  (e)  + ,~h.)  I 


P.(A.  .)?.(A.,  )P,  (A.  ,)PJA,  .) 
) - .1  _iJ J jk  k .11  k ki 

e P.(A..)P  fA,  .)P.  (A.,  )P.  (A.  .) 
1 ji  j kj  k jk  k ij 


■’j<"ij>''k<''jk>hAi^‘’i<*kj> 


Now  D can  be  written  as 
e 


-1  -1  ^^ii^^k^^ii^ 

”c a;(a.J)p,(a;^) 


p;(A.;)pX;i 
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and  hence 


Next  writing  C as 


" vTJCe)  + y‘'(e)  + Ylk<f)Fj(E)  . 


where 


, . . fi<''ii>h(*ik)^<\i>‘“k<\i)fk<^i>‘'k^k> 


it  is  seen  that  by  setting  i,j,k  equal  to  the  three  cyclic  permutations  of 


1,2,3  we  must  have 


F^(e)F2(e)F2(e)  = 1 


Hence  i,j,k  can  be  chosen  such  that 


< yll(c)  + Y23(e)  + y'JCe)  • 


Nov  it  is  easily  verified  that  the  maximal  root  of  the  equation  (2.4),  v:hich  is 


real  and  not  smaller  than  1,  is  an  increasing  and  continuous  function  of  both  the 


coefficients  and  . Thus  by  Lemma  1 there  is  for  every  c > 0 a 3-state 


RFA  for  v/hich  the  error  probability 


P <l-r  +E, 
e 


(2.5) 


where  r is  the  maximal  root  of  the  equation 


(1-X)"  - (I-X)Cq  + Dq  = 0 , 


(2.6) 


c _ -1  a.  -1  -1 

^0  ^12  ^ ^23  ^13  , 
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^ r -1  . -1  -1  -1  -1  ^ -1, 

Dq  max{Y  ^2  ^23  ’ ^12  "^13  ’ ^23  '^13^  ’ 

Clearly  (1/3)Cq  < (1/2)Dq  and  since  (l/2)Dg  ^ 1 we  must  have  ((1/3)Cq)  < ((1/2)Dq)^  . 
Hence  the  maximal  root  of  the  cubic  equation  (2.6)  is  given  by 

1/2 

r = 1 + 2 ((1/3)Cq)-^^  cosh(l/3)(()  , 

-3/2 

where  cosh  tfi  = 1/2  Dq((1/3)Cq)  and  the  bound  (2.2)  follows  from  (2.5).  The 

3 9 

simplified  bound  (2.3)  can  be  obtained  by  increasing  until  ((1/3)  C^)  =((1/2)  D^)  , 

the  maximal  root  of  (2.6)  thus  becoming 


r = 1 + 2((1/2)Dq)- 


3.  COUNTEREXAMPLE  TO  TREE-CONJECTURE. 

Consider  a K-hypotheses  problem  and  assume  ^or  simplicity  that  the  support 

of  each  of  the  distributions  is  same.  With  each  RFA  (A,A,d)  we  can  now 

associate  a graph  F with  vertices  corresponding  to  states  of  the  RFA  and  with 

an  arc  joining  vertices  i and  j if  and  only  if  ^ ® • (This  property 

does  not  depend  on  the  hypothesis  because  of  our  assumption.) 

* 

Let  r.  > 0 , be  the  class  of  all  e-optimal  RFA,  i.e.,  all  m-state  RFA 

(A, A ,d)  such  that 


P^(A,A  ,d)  i inf  P^(A,A  , d)  + e 


It  has  been  conjectured  by  Cover  [2]  that  for  every  c > 0 the  class  always 

contains  a RFA  wi.'ose  graph  F is  a tree.  This  is  indeed  true  for  K = 2 ([3]) 
and  a plausible  heuristic  argument  can  be  given  for  such  a structure  even  for 
K > 2 . Unfortunately,  as  v.'e  are  going  to  show  in  this  section,  the  conjecture 


is  false  already  for  K = 3 . We  do  this  by  exhibiting  a nontrivial  3-hypotheses 


problem  and  a 3-state  RFA  with  a triangular  graph  P , which  is  strictly 
better  than  any  3-state  RFA  whose  graph  is  a tree. 

Let  X = {1,2,3,4,5,61  , let  p,q,r,s  be  positive  numbers  such  that 


2p  + 2q  + r + s = 


■1  r P 
s q 


Define  the  three  distributions  pj^.P2>P2  follows: 


P,(X) 


k 2 


X 

1 2 

3 4 

5 6 

p q 
1 

p q 

s r 

q p 

r s 

p q 

s r 

q p 

q p 

Consider  now  a RFA  (A, A ,d)  with  the  state  space  S = {1,2,3}  , d(i)  = , i€S 


and  the  graph  P the  tree  (I) — (2) — (j)  . The  matrix  R for  this  RFA  is  the  same 
as  that  on  page  8 with  (i,j,k)  = (1,2,3)  . Since  by  Lemma  ] the  error  probability 
is  determined  by  the  maximal  eigenvalue  p of  R and  p is  always  at  least  as 
large  as  the  smallest  row-sum,  in  order  to  minimize  p we  are  forced  to  choose 
A as  follows: 


^12  ~ ’ ^21  ^ ’ ^23  ^ ’ ^32 


wi\ 


The  matrix  R then  becomes 


Writing  its  characteristic  equation  again  as 

(1-A)^  - (l-X)C  + D = 0 

we  have 

C . 3(i)^  , D = (1)'  (i  + £)  . 

There  are  two  other  3-state  RFA  whose  graph  F is  a tree,  one  with  the  graph 

(2) (l) (3)  and  one  with  the  graph  (I) (5) (|)  . By  the  same  reasoning 

as  before  we  are  forced  tc  choose 

A^2  = t2}  , A^^  = {1}  , A^2  = {4}  , A^^  = {3} 

for  the  fonner,  and 

Ai3  = {4}  , A33  = {3}  , A^3  {6}  , A32  = {5} 


for  the  latter.  The  matrices  R are 


and 


1 qs  5. 

■*■  > ";rcr  > ^ 

pr  p 


£I  1 3. 

ps  ’ ’ p 


£ 

P 


£ 

P 


, 1 


respectively,  where  we  omitted  the  5's  for  the  sake  of  simplicity.  Hence 
the  coefficient  of  their  characteristic  equations  are  again  given  by  (3.1). 
Now  consider  a 3-state  RFA  (A,^  ,d)  with 


^12 

= {2} 

’ ^21 

(1)  , 

^13 

= {4} 

hj  ■ 

for 

all  i?4j 

and 

d(i) 

= H. 
1 

is  a triangle. 

The  matrix  R 

is 

31  ~ ^ ’ 23  ~ ’ 32  ” ^ ^ ’ 

, i = 1,2,3  . The  graph  T of  this  RF.^^ 


£ p+2s  ^ p+2r 

p p+r+s  ’ p p+r+s 


£ p+2r  ^ p+2s 

p p+r+s  ’ p p+r+s 

_q  p+2s  £ p+2r  ^ 

p p+r+s  ’ p p+r+s  ’ 


and  che  coefficients  of  its  characteristic  equation 


and 


p p+r+s  p+r+s  p p+r+s 


Comparing  these  expressions  with  (3.1)  we  see  that  C . , < C and 

triangle  tree 


with  r and  s suitably  chosen  also  D . , < D . (Choose,  fo  ' 

triangle  tree 


instance  r = 10  ^p  , s = 10  ^p  . Then  /D  , , = lo'^) . Since  the 

tree  triangle 


maximal  eigenvalue  Increases  with  both  C and  D we  conclude  from  berama  1 
that  the  best  "tree"  RFA  has  an  error  probability  strictly  larger  than  this 
"triangular"  RFA. 


4.  MINIMAX  THEOREM  FOR  FINITE-MEMORY  PROBLEMS. 


Let  = (^j^,  . • . , be  a probability  distribution  on  the  set  of  hypotheses, 


let  (A, A ,d)  be  RFA,  and  let  this  time  the  error  probability  be 


K 


P^(1;(A,A  ,d))  = I TT  pf‘"^(A,A  ,d)  . 
e k=l  ® 


Looking  now  at  the  problem  as  a two-person  zero-sum  game,  where  the  1st  player 
(Nature)  chooses  and  the  2nd  player  (Statistician)  chooses  (A, A ,d)  it 
is  natural  to  ask  whether 


inf  sup  P (t^;(A,A  ,d))  = sup  inf  P (jn;(A,A  ,d)) 


(A, A ,d)  -rr 


(4.1) 


^ (A, A ,d) 


Now  if  K = 2 then  it  is  known  [3]  that 


inf  P (tt,  (A, a ,d))  = 
e — 


m-1  1/2 

2(.i.2Yi2  > - 1 


m-1  , 

^12  ' '■ 


(4.2) 


where  1^  given  by  (2.1)  . Hence 


sup  inf  P = 

e m-1 


. m-1  1/2 

^12  " ^ . n~  . -1 

V m-1  '' 


=!-(!+, 


^12  - 1 


'12 
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On  thp  other  hand  by  Lemma  1 


Inf  sup  = 1 - (sup  p(A,A,d)) 


^nd  for  K = 2 It  is  easily  seen  that 


p(A,A  ,d)  = 1 + 


1/2 


However  it  has  also  been  shown  in  [3]  that 

sup  = — pr 

P^(d  ^H^))b2(d‘ 

and  hence  (4.1)  is  indeed  true  for  K = 2 . 

Conjecture ; (4.1)  is  also  true  for  K > 2 . 

Comment : Since  an  analog  of  (4.2)  for  K > 2 is  not  available  at  present  the 

above  reasoning  cannot  be  applied  to  prove  the  conjecture.  However,  since  the 
number  of  hypotheses  is  finite  (4.1)  would  follow  if  one  could  show  that  the 
set  of  all  vectors 


(P^^^A.A  ,d),.  ..,P^‘^^(A,A  ,d))  , 
e e 

where  (A, A ,d)  runs  through  all  m-state  RFA,  is  convex.  This  is  indeed  so  for 
K = 2 , unfortunately  we  have  not  been  able  to  prove  this  even  for  K = 3 . 


15  - 


APPENDIX  A 


A Formula  For  A Stationary  Distribution 
Of  A Finite  Markov  Chain. 


Let  P = [P^j]  be  an  m x m stochastic  matrix,  let  g = (S,E)  be  an 


oriented  graph  with  the  set  of  vertices  S = and  the  set  of  arcs 

E C S X s defined  by 


(i.j)6  and  > 0 . 


Let  i€S  be  a vertex  of  g.  Then  a ’"'.rtex  j£S  such  that  (i,j)£E 
is  called  a successor  of  i 
that  each  i, 


A sequence  of  vertices  (ij^,i2»  • ■ • ,i^)  such 


k+1 


is  a successor  of  1 


, k = l,...,n-l  , is  called  a path. 


If  i,  is  also  a successor  of  i the  path  (i,  ) is  called  a cycle. 

1 n 1 n 


Consider  now  a subgraph  f = (S,F)  , where  FC  E with  the  following 
properties. 

1)  each  vertex  i€  S has  at  most  one  successor. 

2)  f has  no  cycles. 

3)  f is  maximal,  i.e.,  no  further  arcs  can  be  added  without  violating 
1)  or  2). 

We  will  call  such  a subgraph  a confluence.  Notice  that  each  confluence 
has  exactly  one  vertex  with  no  successor.  We  will  refer  tp  this  vertex  as  a 
sink. 

With  each  confluence  f = (S,F)  we  associate  a positive  number 


p(f)  = 


n 


(i,j)€F 


We  now  have  the  following  theorem: 
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Theorem:  Let  P be  a transition  probability  matrix  of  a homogeneous  Markov 

chain,  g be  the  graph  defined  above,  let  be  the  set  of  all  confluences 

with  sink  1 £ S . 

If  P has  an  Invariant  distribution  » • • • )P  then 

i m 1 m 


“ C I p(f)  , i € S , 
f 


(A.l) 


where  C > 0 is  a normalizing  constant  determined  from  g,  + ...  + u =1. 

i m 

Remark:  Notice  that  the  formula  (A.l)  gives  as  a sum  of  products  of  the 

off-diagonal  entries  of  P , each  product  contains  exactly  m - 1 different 
entries  and  no  two  products  contain  the  same  set  of  this  sense, 

the  representation  of  g^  is  unique.  Notice  also,  that  if  all  off-diagonal 


entries  of  P are  positive  then  g^^  is  a sum  of  exactly  m 


m-2 


products . 

Thus,  although  the  formula  is  certainly  of  theoretical  Interest,  its  application 
for  computing  the  stationary  distribution  is  likely  to  be  limited  to  cases, 
where  a majority  of  P^j'®  zero. 

Example : Let 


P = 


The  graph  g is 


.5  .2 

.3  0 

.3  .1 

.2  .4 

.1  .7 

.2  0 

0 .5 

0 .5 

(D^:»©^=%0 

and  the  confluences  together  with  the  number  p(f)  are  as  follows: 


17  - 


Wftlf 


Proof  of  the  Theorem;  We  will  show  that  (A.l)  satisfy  the  equations 


HI 

M . = y p .p . . , i=i, . 

1 1 ij 

-j  i_=i  j 


or  equivalently 


'J  ih  ■ ill  ” ■ 


Let  h = (S,H)  be  an  arbitrary  subgraph  of  g , let 


P(h)  = f’  p..  , 

(i,j)€  H 

and  let  h + (i,j)  denote  a subgraph  obtained  from  h by  adding  or  removing 
the  arc  (i,j).  Next  let 


= {f  + (j,i)  : f€  S - {j)]  , 


= {f  + (i,j)  : i€  ^^,i€S  - {j}}  . 


(A. 2) 


If 


.,p^  is  given  by  (A.l)  then  for  any  j£  S 


4 I P44  = I an'l 


J i=l  "J  h£A. 


I P4P44  = I P(h)  . 

i=l  J h€B. 


Thus  (A, 2)  will  follow  if  we  show  that  A.  = B.  . To  this  end  let 

J J 


h C A^  , h = f + (j  ,i)  , 
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let  k be  a vertex  contained  in  the  path  (i,...,j)  whose  successor  is  j. 


If  the  arc  (k,j)  is  removed  then  h becomes  a conluence  with  sink  k 


since  (k,j)  was  an  arc  of  confluence  f and  thus  could  not  have  any  other 


successor  than  j . Hence  h can  be  written  as  f + (k,j)  , f'€  <})j^  and 


therefore  ^ . Conversly,  if  ^ ^ ^ j » say  h «=  f + (k,j)  , then 


f'f  (j),  and  by  removing  the  arc  (j,i)  with  i being  a successor  of  j con- 


tained in  the  path  (j,...,k)  we  conclude  that  h - (j,i)£(J>j-  Hence  ^ 


and  the  proof  is  complete. 


APPENDIX  B 


A Generalization  of  a Lemma  of  Yakowitz  ([5]). 


Lemma : Consider  K finite  regular  Markov  chains  with  state  spaces  , 


transition  probabilities  [Pj^(i-vj)]  , and  stationary  distributions 


k = 1,...,K  . Link  these  chains  together  by  allowing  transition  between  S, 


and  , k - 1 K - 1 , via  a pair  of  atatea  ,^£  S 


with  probabilities 


^^\,k+l  ^ \+l,k^  \,k+l  ’ 


^^®k+l,k  ^ \,k+l^  ■ \+l,k  ’ 


and  changing  the  original  transition  probabilities  k+1^ 


Vl<\+l,k  * Vl,k>  »=“rdlngly. 


If  the  new  chain  with  state  space  S = U • • • U is  regular  then  its 

X K. 


stationary  distribution  is  given  by 


a£  S^^p(s)  - Cp^(s),n^ 


n n.  . ,u.(e.  , ,)  , 


j=k+l 


j,j-r 


k = 1,...,K  , 


where  C > 0 is  a normalizing  constant. 


Proof  (by  induction  on  K) 


(i)  Let  K = 2 . We  have  for  the  ori'jinal  two  chains 


s£  S r^U,(s)  = I P (r->-s)y  (s)  , 
r€S^ 


s€  S2^y2^®^  “ 1 P2(r^)y2(s)  , 


and  for  the  new  chain  S^2  ~ S^U  S^  , 


s€S^,s  ^ e^2=^^'l2^®^  ""  ^ P^^(r->s) y^2^®)  > 

r€Sj^ 

s€  S^.s  i e2^=^Pj^2^®^  ^ I P2(r->s)y^2(s)  » 


r€S^-{e^2^ 


+’21>'l2<'21>  ' L ^('^•*=>‘'l2<=' 


since  ^21*^12^^21^  ” ^12*^12 *'*12^  equating  flows.  Similarly 


^12(^2]^)=  I P2(r->s)y^2(®)  • 

f€  S2 


Hence  if  s € S^  then  y^(s)  and  satisfy  the  same  system  of  linear 

equations  and  consequently 


seSj^^U^^(s)  = a^y^(s)  , 

s € 82=^  • 


In  particular 


*^12^^12^  ^1*^1^®12^  ’ *^12^^21^  ^2*^2^®21^ 


and  since  ''^12*^12^^12^  ~ ^21*^12^^21^  must  have 


h ’21  “2<°2I^ 

^*2  ’*12  '*l'®12^ 
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Since  = 1 this  implies  a^^  = • 


(ii)  Let  the  lemma  be  true  for  S = S (J-.-LJ  S and  form  new  chain  SUS 

1 K.“i  K 


Denoting  m the  stat.  distribution  of  S and  p that  of  S U we  have 

K 


by  part  (i) 


S6S^„M  . C 


(B.l) 


By  induction  hypothesis 


K-2 


'*  t ^ I/)  i.LI  Jll) 


'K-1,K 


K-1'  K-i,K' 


and  if  s € S,  C S 
k 


k-1 


K-i. 


p'(s)  = C'u  (s)  il  n.  p.(e.  Jl  ir.  . . p.(e.  . J 

k d,j+]  J J,J+i  j.j-i  J j,j-r  . 


Substitution  into  (ii.l)  gives  the  desired  formula  (with  the  proportionality 
constant  CC ' ) . 
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I ■' 


M 


'-'im 


: i 


1 1 


1 1 

4 


